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METHOD AND SYSTEM FOR AUTOMATIC CORRECTION 
OF MOTION ARTIFACTS 

FIELD OF THE INVENTION 

The present invention relates to methods and systems for improving interlaced 
video and images taken from endoscopic or other surgery, sports still frames, or other 
video. In particular, the methods and systems of the present invention relate to 
improving interlaced video and images in which the image recording device, subject, or 
both are moving independently. The present invention more particularly relates to 
methods and systems for automatically correcting artifacts caused by camera motion 
and artifacts caused by subject motion. 

BACKGROUND OF THE INVENTION 

Television and video technology has long relied on trickery to fool the human eye 
into believing that the television or video signal accurately reflects the events occurring. 
For instance, conventional video technology such as the NTSC (National Television 
Standards Committee) standard uses "interlaced video," in which a single image or 
frame comprises two fields that are taken 1/60 second apart. Each frame contains 525 
scan lines divided into two fields. The first, or odd, field comprises the odd numbered 
lines (e.g., 1, 3, ... 525) while the second, even field forms the even numbered lines 
(e.g., 2, 4, ... 524). These two fields, during display of the frame they form, are 
interlaced so that the odd numbered scan lines are inserted between the even 
numbered scan lines, much like the interlacing that would occur when one interlaces 
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the fingers of one hand with the fingers of another. Since each frame comprises two 
interlaced fields (even and odd) each captured 1/60 second apart, the frames 
themselves are captured at a rate of 30 frames per second. 

In most video applications, the very short delay (1/60 second) between capture 
5 of fields means that even if the subject moves during filming, any artifacts of motion will 
be virtually undetectable. (An example of motion artifacts can be seen in old silent 
movies, which operated at fairly low frame speeds.) The high rate at which the recorder 
captures frames and the slight time separation between the fields within each frame 
results in minimal blurring in the video. Even when the subject being video-recorded is 
io in motion, for example, a moving car or a runner, such motion artifact will be 
substantially undetectable to the human eye in the captured frame. Additionally, in 
many applications, humans tend to ignore even detectable motion artifacts in video 
because a particular motion artifact (for example, blurring because of subject 
movement) is often quickly replaced with another frame or series of frames missing that 
15 artifact. Thus, in some applications, the speed of frame capture and the continual 
refreshment of interlaced fields is sufficient to avoid noticeable blurring of video images. 

However, in many applications, the rate at which fields are scanned and frames 
are captured is not sufficiently high to prevent motion artifacts from causing image 
distortion problems in the displayed video. Applications in which image distortion 
20 problems may be caused by motion artifacts include, for example, video and images 
taken from endoscopic or other surgery, sports still frames, or other video in which the 
camera and subject are moving independently. U.S. Patent No. 5,191,413 to Edgar 
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gives a practical example of this. Edgar states that "if a subject were to move a hand 
during the capturing process, sequential fields will be generated which will capture the 
hand in two distinctly different positions. However, operation of typical interlace 
systems call for the two fields to be continuously refreshed alternately on the display 
5 screen. The results of this may be the appearance of the hands engaging in a jittery or 
shaking motion at the frequency of 30 times a second giving rise to highly undesirable 
images." This phenomenon is also illustrated in U.S. Patent No. 5,329,317 to 
Naimpally, et al. 

5 These motion artifacts are particularly pronounced when the video in question 

lo has been magnified. For example, videos are often taken and magnified in endoscopic 
I or laparoscopic surgery. Video images are taken by a high resolution camera coupled 
u to an endoscope optic or laparoscope optic. Such images are magnified tremendously 
1 by the scope optic and, as a result, the captured images are extremely sensitive to 
t motion. Thus, a small movement in an image field results in a much larger change in 
5 15 the viewed field, and such magnified motion appears to be more global than local. In 
addition, because of this magnification, such motion is exaggerated by the time 
separation between the two field components of the captured frame. 

Motion effects in captured frames are created in at least two different ways, each 
of the at least two ways resulting in different types of motion artifact. One type of 
20 motion effect is generated by movement of the endoscope camera by the surgeon. 
Endoscope movement can result in a "uniform motion-related error" between the odd 
and even fields. This type of motion effect is known generally as "linear artifact." As 
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both fields are captured, endoscope movement causes a nearly identical image to be 
shifted in direct proportion to the velocity of endoscope movement, thus producing 
linear artifact. A second type of motion effect is created by movement within the image 
field of the camera. Such motion may be due to movement of surgical tools by the 
5 surgeon or by movement of the patient, such as with breathing. This type of motion 
effect is localized to the region of the image in which movement of the surgical tools or 
the patient tissue is being viewed, and is known as "regional artifact." The substantial 
magnification by the endoscope or laparoscope optic exacerbates the distorting effect 
of motion in a captured image caused by both linear artifacts and regional artifacts, 
jjio in sensitive applications, such as surgery, it is important to provide the most 

rii stable and artifact-free image possible. Efforts have been made in the past to correct 
H for these motion artifacts. For example, video printers available from manufacturers 
« such as Sony have "motion check" firmware. That firmware looks for motion artifacts 
5 developed within a particular image and alerts the user when it finds them. The user 
9i5 can then correct for the motion artifact. The correction usually involves dropping one of 
the two fields forming the image, which greatly reduces the vertical resolution of the 
video being displayed or printed. Another conventional technique for correcting for 
motion is to drop one of the two fields and replace the discarded field by repeating the 
remaining field. This results in an image that exaggerates only half the captured 
20 information, resulting in lower resolution. Also, some commercial software applications 
have motion correction features that can be performed by the user, although the 
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features are often difficult to implement by those not technically versed in its use. 

Adobe has such software. 

Another approach to correcting for motion artifacts has been to compare the 
difference between pixels in two adjagent fields to a fixed threshold value. If the 
5 threshold is exceeded, then the value in one pixel is replaced. The replacement value 
may be determined by averaging the value of pixels in adjacent lines. If, however, the 
difference between pixels does not exceed the fixed threshold, no action is taken to 
change a pixel value. This process is repeated for each pixel in each line of each 
frame. An example of this approach as applied to motion artifacts within an endoscopic 
10 image field is described in U.S. Patent No. 5,877,81 9 to Branson. 

Conventional methods for correcting for motion artifacts in highly magnified 
videos do not result in the highest qualify picture. Moreover, they often require users 
with specialized skills, and lack flexibility. Thus, there is a need for a method and 
system for correcting for motion artifacts in highly magnified videos that yields a high 
15 resolution image. There is also a need for such a method and system that is dynamic 
with respect to pixel value thresholds so as to increase flexibility in further pixel value 
analysis and replacement. Moreover, there is a particular need for such methods and 
systems in sensitive applications, such as surgery. 

20 SUMMARY OF THE INVENTION 

The present invention provides methods and systems for improving video and 
images taken from endoscopic, or other medical and surgical procedures, sports still 
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frames, or other video in which the image recording device, subject, or both are moving 
independently. An example of an embodiment of the present invention described 
herein is a method for automatic correction of motion artifacts in an interlaced video 
image captured by an image recording camera. Such an embodiment may include 
capturing a complete frame of an interlaced video image, automatically correcting for 
camera motion, automatically correcting for subject motion, and displaying an image 
corrected for camera motion and subject motion. 

In an embodiment, the method of the present invention involves first determining 
whether the image has been affected by camera motion. Since an entire image is 
affected by linear artifacts caused by camera motion, such camera motion artifacts 
must be removed from the image as a whole before the image can be examined for 
regional motion. Camera motion is identified and measured using auto-correlation of 
the odd and even raster fields. A two-dimensional vector is then calculated in order to 
provide optimal correction for camera motion, after which the even field may be 
repositioned into its correct location relative to the odd field (or vice versa) to correct for 
camera motion during the 1/60 second interval between capture of the two fields. This 
aspect of the invention corrects for situations such as "linear artifact" in which the entire 
camera itself moves relative to the viewed subject (or vice versa). 

In another aspect of the invention, a method is provided for automatically 
identifying segments within the particular video image that contain subject, or regional, 
motion artifacts. Such a method of the present invention first determines where in an 
image subject motion has occurred. This may be done by convolving the three red- 
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green-blue (RGB) color components of the odd field to compute or predict an even field. 
Differences between the measured even field and the predicted even field indicate 
regions of motion within the image field. When the difference between the predicted 
even field and actual even field is small, the difference can be attributed to subject 
detail and is left uncorrected. When the difference between the predicted even field 
and the actual even field is large, subject, or regional, motion within the image field is 
identified. 

The region of the image to which correction needs to be applied is then 
determined. One method of determining the region for correction is to form a subject 
motion map. In one implementation, a subject motion map may be generated by 
squaring the differences between predicted even fields and actual even fields. 
Squaring the differences in pixel values between fields eliminates any positive or 
negative direction associated with those values, and makes the result less sensitive to 
small errors. The resulting subject motion map may then be compared to a threshold to 
create a binary image delineating the region of the image in which regional motion 
artifacts have occurred. 

Identified regional subject motion J is then corrected. In one method, correction 
can be accomplished by computing the even field from the odd field in the region of the 
image where the subject motion map indicates low even field prediction accuracy. By 
comparing identified regions of local artifact to the number of neighboring pixels above 
a set luminescence threshold, unwanted pixels may be eliminated one at a time. 



Express Mail No. EL 146 846 537 US 

Each region in the subject motion map where unwanted pixels have been 
eliminated may then be replaced but without the blurring caused by regional motion 
artifacts. In one method, auto-correction replacement for eliminated pixels can be 
accomplished by detecting adjacent pixels in two different directions. Such a two 
dimensional vector computed from non-blurred pixels at the boundaries of eliminated 
subject motion regions can be used to adjust the map by translating a sharp image 
therefrom into the region of subject motion. 

A finished, corrected image can then be computed. In one method according to 
the present invention, a finished, corrected image is computed by using the adjusted 
map to indicate where to modify the original NTSC frame. Selective correction of the 
original frame may be accomplished by computing an even raster from the odd raster 
where the map indicates subject motion is greatest. After a finished image, 
automatically corrected for both camera motion and regional motion, is computed, the 
image is displayed for viewing. 

One advantage of embodiments of the present invention is that motion artifacts, 
whether linear or regional, are automatically corrected. An advantage of regional field 
synthesis duplication as in the present invention is that there are no visible edge effects 
in the viewed image. In other words, regions of subject motion have smooth edges, 
rather than the jagged edges that often result from over-correcting. 

Another advantage of automatically correcting only for regions that need 
correction as in the present invention is a high resolution image. Pure regional field 
duplication results in the loss of some image detail from discarding the even raster 
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pixels in the region of subject motion. Yet, in embodiments of the present invention, 
because pixels are discarded only in the regions needed, much fewer pixels are 
discarded than in systems that discard an entire field of pixels. Thus, the method and 
system of the present invention provide a flexible means for automatic correction of 
5 motion artifacts in highly magnified videos that result in the highest resolution-quality 
picture. 

Embodiments of this invention automatically detect the type of artifact 
encountered in a captured frame on a pixel by pixel basis. Systems according to the 
present invention automatically identify whether the motion artifact is linear artifact 
Bio caused by relative camera movement or regional artifact in the image field caused by 
& subject (surgical instrument or patient) movement. By utilizing different parameters for 
H detecting different types of motion, corrective action is taken only for that type of motion 
p detected. 

5 After automatically identifying the type of artifact, embodiments of this invention 

? 15 then automatically correct the frame image using the correct compensation for the 
particular artifacts encountered on each pixel of the frame. These benefits avoid any 
need by the surgeon to determine at the time of video capture whether motion 
correction is needed, and thus avoid prolonging surgical procedure time. In addition, an 
advantage of this system is that it allows the surgeon to capture still images without 
20 having to concentrate on assuring that the camera, the surgeon, surgical tools, and the 
patient are all still at the same instant each time an image is captured. Images can be 
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captured essentially at random without concern for motion artifacts caused by 
movement of any of these potential sources of artifacts. 

Another advantage of the present invention is that embodiments can be used in 
the operating room, eliminating the time otherwise required to make post-procedure 
image corrections. Non-automatic motion correction procedures typically involve the 
surgeon manually editing captured images by manipulating images with functions such 
as "motion up" and "motion down." The automatic correction for motion artifacts of this 
invention eliminates any need for manual editing. 

Further, the minimal time required to apply the automatic motion correction of 
this invention renders practical real-time image correction. This process can be 
applied, in real-time, using dedicated image processing equipment with the process 
implemented in dedicated circuitry or special purpose computational elements. 
Alternatively, this process can be applied by using a general purpose computer to 
compute corrections for motion artifacts using the process implemented in software. 

Methods and systems for automatic motion correction according to the present 
invention may be used in a wide variety of applications in which video and images are 
taken from endoscopic or other surgery, sports still frames, or other video in which the 
camera and subject are moving independently. The present invention may be used 
with various video signal formats, including NTSC, PAL, and other sources of interlaced 
video signals. 

Therefore, the method and system of the present invention provide for automatic 
motion correction in video and images in which the camera, subject, or both move 
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independently that results in a high quality-resolution image. This is accomplished by 
maintaining primary information from both raster fields in an interlaced frame. The 
method and system of the present invention automatically and differentially detect linear 
motion artifacts caused by camera movement and regional motion artifacts caused by 

5 movement of surgical tools or the patient within the image field, both types of motion 
which may be occurring simultaneously. Using the method and system of the present 
invention, each type of motion artifact may be automatically and differentially corrected. 

Those of ordinary skill in the art will appreciate the advantages and features of 
the present invention as described above and as is apparent from the detailed 

10 description below. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIGURE 1 is a schematic diagram illustrating a system for practicing automatic motion 
correction of the present invention. 
15 FIGURE 2 is a flow diagram illustrating operation of an embodiment implementing the 
automatic correction system of the present invention. 

FIGURE 3 is a more detailed flow diagram illustrating options for applying an automatic 
correction system of the present invention in operation of the embodiment depicted in 
Figure 2. 

20 FIGURE 4 is a flow diagram illustrating methods for automatic motion correction of the 
present invention as embodied in the system depicted in Figure 1 . 
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FIGURE 5 is a more detailed flow diagram illustrating methods for automatic motion 
correction of the present invention as embodied in the system depicted in Figure 4, also 
showing steps to achieve automatic correction for camera motion and automatic 
correction for subject motion. 
5 FIGURE 6 is a photograph depicting image distortion due to linear artifacts caused by 

camera motion. 

FIGURE 7 is a photograph of the image of Figure 6 after automatic motion correction of 
camera motion linear artifacts. 

FIGURE 8 is a photograph depicting image distortion due to camera motion and 
Sio regional subject motion, as identified by a subject motion map. 
m FIGURE 9 is a photograph of the image of Figure 8 after automatic motion correction 
H for camera motion and automatic correction for regional subject motion. 

S DETAILED DESCRIPTION 

£ 15 Methods and systems for automatic motion correction according to the present 

w invention may be used in a variety of applications. Examples of applications include 
video and images taken from endoscopic, or other medical and surgical procedures, 
sports still frames, or other video in which the camera and subject are moving 
independently. The present invention may be used with various video signal formats, 
20 including NTSC, PAL, and other sources of interlaced video signals. In addition, the 
present invention may be used with a variety of hardware applications. 
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An example of an embodiment of the present invention is a method for automatic 
correction of motion artifacts in an interlaced video image captured by an image 
recording camera. Such an embodiment includes capturing a complete frame of an 
interlaced video image, the complete frame having a first raster field interlaced with a 
second raster field and both fields comprising pixels. Such an embodiment also 
includes automatically correcting for camera motion, automatically correcting for subject 
motion, and displaying an image corrected for camera motion and subject motion. 

Methods of an embodiment of the present invention also utilize the steps of 
locating optimal correlation values between pixels in the first and second raster fields 
and creating a two-dimensional motion vector from such optimal correlation values. To 
achieve correction for camera motion, this embodiment then creates a synthetic first 
raster field by duplicating the second raster field in the captured complete frame in a 
corrected position according to the two-dimensional motion vector. 

To accomplish correction for subject motion, methods of an embodiment of the 
present invention may also utilize the step of computing a subject motion map to 
identify regions of the captured frame where differences in pixel values between the 
first and second raster fields are relatively large. From a subject motion map, a binary 
subject motion location map may be created to delineate regions of the captured frame 
identifying pixels where the number of neighboring pixels exceeds a pre-determined 
threshold. In this embodiment, the pixels identified in regions of subject motion may 
then be eliminated. Pixels eliminated from regions of subject motion may then be 
replaced, but without blurring, by computing a two-dimensional vector from pixels at the 
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boundaries of eliminated regions. In this embodiment, a finished, corrected image may 
be created by using the adjusted binary subject motion location map to indicate regions 
on the captured frame where subject motion is greatest, and computing a corrected 
second raster field from a corrected first raster field in regions where subject motion is 
greatest. Methods of the embodiments described above may also utilize the step of 
displaying the image corrected for camera motion and subject motion on a video 
monitor. 

In an application of the present invention in endoscopic surgery, one 
embodiment uses the "Dyonics® Vision 625 Digital Capture System" by Smith & 
Nephew, Inc. of Andover, Massachusetts. The "Dyonics® Vision 625 Digital Capture 
System" ("Dyonics® Vision 625") is designed to be used in the operating room to 
digitally capture intra-operative images. Images are stored in standard image file 
format on transportable media for long term archival, retrieval, or printing. 

An embodiment of automatic motion correction for video images according to the 
methods and systems of the present invention is described using the "Dyonics® Vision 
625," for example, in endoscopic surgery, in the steps below. FIGS. 1-4, in particular, 
depict operation of such an embodiment for practicing automatic motion correction as in 
the present invention. 

1 . Capture a complete frame (Frame 1 ). 

The methods and system of automatic motion correction of the present invention 
operate on an interlaced video image comprising a complete frame having both even 
and odd fields captured. Referring to FIG. 5, for example, a complete NTSC frame 501 
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may be captured using a standard video capture device. In an embodiment using the 
"Dyonics® Vision 625" system 100, endoscope optic 101, as seen in FIG. 1, detects a 
video image in which both camera motion and regional subject motion are automatically 
detected and corrected. Referring to FIG. 1, an interlaced video signal is transmitted 
from endoscope optic 101 through camera control unit 102 to digital capture unit 103, 
where a full frame, interlaced video image is captured. 

Digital capture unit 103 may have a keyboard 104 attached, which may be used 
to activate various modalities of the system, such as "freeze" and "capture," as seen at 
references 306 and 308, respectively, in FIG. 3. Live video images detected by 
endoscope optic 102 may be viewed (305a, 305b) as displayed on video monitor 105. 
In operation, the surgeon may activate a "freeze" signal 110 at the camera control unit 
102 to freeze (306) a live video image (305a) and display the frozen image (307) on 
video monitor 105. As the surgeon views a live image (305a), the image may be 
captured by activating a "capture" signal at the camera control unit 102, the image 
being saved (309) to the internal temporary storage 112 of digital capture unit 103. By 
activating "bypass" mode 330, the digital capture unit 103 is deactivated, or suspended 
from application to the live video images being viewed, and live images are displayed 

directly on video monitor 105. 

Automatic correction for camera motion 405 and automatic correction for 
regional subject motion 406, depicted in greater detail in FIG. 5, are operationalized by 
Automatic Motion Correction (AMC) Filter 109 of the embodiment shown in FIG. 1. 
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Automatic correction for both camera motion and for regional subject motion are 
described below. 

2. Perform auto-correlation on a first raster field with respect to a second raster field 
(Frame 1). 

The first automatic motion correction performed by the methods and system of 
the present invention is for camera motion. FIG. 6 illustrates image distortion due to 
linear artifacts caused by camera motion. Correction for camera motion is 
accomplished by performing auto-correlation on a first raster field, either the even field 
or the odd field, with respect to the second raster field, the complementary field to the 
first field, in the captured frame. The auto-correlation function results in a "motion 
vector." Using the "Dyonics® Vision 625" system, the auto-correlation motion vector is 
computed by performing the function "MeasureMotionVector" 502 in FIG. 5. 
"MeasureMotionVector" 502 outputs two values through the argument list SkewX and 
SkewY. To save time, the algorithm uses an iterative search procedure, minimizing a 
3x3 convolution. The best-fit location of the 3x3 result matrix is used to search at a new 
position until an optimal match is found, or the first raster is determined to be greater 
than 1 5 pixels offset from the second raster. 

"MeasureMotionVector" 502 calls one subroutine, "Convolution" 503. The 
purpose of "Convolution" is to save results from successive computations to eliminate 
duplication of computations already performed in previous searches for the optimum 
auto-correlation point. Generally, "CohvolveAt" is a mathematical technique that 

16 
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compares two images by generating a single number for each of various X and Y points 
indicative of the quality of the fit of each test pixel in one raster relative to the reference 
pixel in the other raster. 

In the present invention, "ConvolveAt" convolves the difference of all pixels 

5 between the odd and even rasters for a specified offset. When the specified offset is 
zero for both X and Y, the same pixels in both rasters are compared to each other. 
When the offset is one X and zero Y, the even raster pixel to the right of the same pixel 
in the odd raster is compared. The convolution in "ConvolveAt" measures the sum of 
the square of the differences between the rasters at the specified offset. The optimum 

10 auto-correlation between the odd and even rasters is the value of X and Y which 
produces a minimum difference between the two rasters. By determining the optimum 
auto-correlation points for shifting the test image relative to a reference image, 
convolution creates as sharp a full-frame image as possible. The convolution step 
corrects for linear artifacts caused by camera motion, as shown by reference 405 in 

15 FIG. 4. Yet, image differences between the two raster fields due to regional artifacts 
may still be present at this point. 

The following algorithm applies to auto-correlation on a first raster field with 
respect to a second raster field. However; as will be apparent to those of ordinary skill 
in the art, the automatic motion correction method and system of the present invention 

20 are also readily utilized in applications that do not employ this convolution technique. 

void MeasureMotionVector(unsigned char *Fields, long *SkewX, long *SkewY) 
{ 

unsigned long Data[33][33] = {0}; 
25 int x, y, NextX, NextY; 

17 
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unsigned long Center, t, I, b, r; 
double Szy2, Szy, Sz, A, B, DeltaY; 

NextX = 0; // kick start value! 
5 x = 1 ; // kick start value! 
NextY = y = 0; 

while((x != NextX) || (y != NextY)) 
{ 

10 x = NextX; 

y = NextY; 

Center = Convolution(Data, Fields, x, y); 
t = Convolution(Data, Fields, x, y-1); 
15 I = Convolution(Data, Fields, x-1 , y); 

b = Convolution(Data, Fields, x, y+1); 
r = Convolution(Data, Fields, x+1, y); 

if(Center > t) 
20 { 
n NextX = x; 

& NextY = y-1; 

Center = t; 

I } 
n25 if(Center>l) 

m { 

i"i NextX = x-1; 

iT NextY = y; 

7" Center = I; 

JUo } 

if(Center > b) 

II { 

y NextX = x; 

£ NextY = y+1; 

O35 Center = b; 

C } 

if(Center > r) 
{ 

NextX = x+1; 
40 NextY = y; 

} 

Center = Convolution(Data, Fields, x, y); 
if(x ==15) break; 
if(x == -15) break; 
45 if(y == 15) break; 

if(y == -15) break; 
if(!Center && !t && !b) 
{ 

*SkewX = 0; 
50 *SkewY = 0; 

return; 
} 

} 
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// using least squares, interpolate Y from t(op), Center, and b(ottom) 
// note: Center is at zero pixels offset, t is -2 and bottom is 2 

// using notation: z is a function of y... and the 2nd order poly coefficients a, b, c 
5 // where z = ay A 2 + by + c 

// f(a, b, c) = Sigma(((ay A 2 + by + c) - z) A 2); 

// minimize f(a,b,c) 

// take partial derivatives: 

// df/da = 2 Sigma yy((ayy + by + c) - z) 
10 // 1a) df/da = 2 Sigma (ay A 4 + by A 3 + cy A 2 - zy A 2) 

// df/db = 2 Sigma y((ayy + by + c) - z) 

// 2a) df/db = 2 Sigma (ay A 3 + by A 2 + cy - zy) 

15 // 3a) df/dc = 2 Sigma (ay A 2 + by + c - z) 

// setting the partials to zero to find the minima... 
// 1b) Sigma zy A 2 = aSigma y A 4 + bSigma y A 3 + cSigma y A 2 
// 2b) Sigma zy = aSigma y A 3 + bSigma y A 2 + cSigma y 
20 // 3b) Sigma zz = aSigma y A 2 + bSigma y A 1 + nc (note n=3) 

S II compute the matrix coefficients: 

m Szy2 = t * 4.0 + Center * 0.0 + b * 4.0; 

Szy = t * -2.0 + Center * 0.0 + b * 2.0; 

3:25 Sz = (double)t + (double)Center + (double)b; 

I:; // Sy4 = 32 = 16.0 + 0.0 + 16.0; 

fl II Sy3 = 0 = -8.0 + 0.0 + 8.0; 

ft //Sy2 = 8 = 4.0 + 0.0 + 4.0; 

^ // Sy = 0 = -2.0 + 0.0 + 2.0; 

y! // the matrix is: 
* ;: // 1c) Szy2 = a*32.0 + b*0.0 + c*8.0 
O // 2c) Szy = a*0.0 + b*8.0 + c*0.0 
Hh // 3c) Sz = a*8.0 + b*0.0 + c*3.0 

U35 

O // To find the y location of the 

// parabolic vertex: minimize dz/dy = 2aDeltaY + b 
// so DeltaY = -b/2a, therefore we only need to solve the matrix for a and b. 
//from eqn 2c: 
40 B = Szy / 8.0; 

// now with two eqns in 2 unknowns: 
//1d) Szy2 = a*32.0 + c*8.0 
//3d) Sz = a*8.0 + c*3.0 
45 // normalize c 

// 1e) Szy2/8.0 = a*32.0/8.0 + c 
// 3e) Sz/3.0 = a*8.0/3.0 + c 
// rearrange 

// 1f) Szy2/8.0 - a*32.0/8.0 = c 
50 //3f) Sz/3.0 -a*8.0/3.0 = c 
// solve for a: 

// Szy2/8.0 - a*32.0/8.0 = Sz/3.0 - a*8.0/3.0 
// Szy2/8.0 - Sz/3.0 - a*32.0/8.0 = - a*8.0/3.0 
// Szy2/8.0 - Sz/3.0 = a*32.0/8.0 - a*8.0/3.0 
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// Szy2/8.0 - Sz/3.0 = a*(32.0/8.0 - 8.0/3.0) 
// (Szy2/8.0 - Sz/3.0)/(32.0/8.0 - 8.0/3.0) = a 
A = (Szy2/8.0 - Sz/3.0)/(32.0/8.0 - 8.0/3.0); 

5 // so finally: 

DeltaY = -B/(2.0*A); 

// report the results to the calling function 
*SkewX = -x; 

10 *SkewY = -y*2 - DeltaY*2; // round instead if truncate 
} 

unsigned long Convolution(unsigned long Data[33][33], 
unsigned char *Fields, long x, long y) 

15 { 

// already computed? 

if(Data[y+16][x+16]) 

return Data[y+16][x+16]; 

Data[y+16][x+16] = ConvolveAt(Fields+640*3*240, Fields, x, y); 
20 return Data[y+16][x+16]; 
j } 

1 unsigned long ConvolveAt(unsigned char *Odd, unsigned char *Even, long x, long y) 

* { 

J25 long i, j, Sigma = 0; 

x += 16; 
y += 16; 

= 30 Odd +=16*640*3 + 16*3; 
Even += y * 640 * 3 + x * 3; 

for(i=0; K240-33; i++) 
{ 

f 35 j = (640-33)*3; 

2 while(j-) 

Sigma += (*Odd - *Even) * (*Odd - *Even); 
Odd++; 

40 Even++; 

} 

Odd += 33*3; 
Even += 33*3; 
} 

45 

return Sigma; 
} 

3. Create a synthetic first raster field by duplicating the second raster field and then 
50 placing the synthetic first raster field into its corrected position (Frame 2). 
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To complete automatic motion correction for camera motion, an intermediate 
frame, substantially transparent to the human eye, is created. Figure 7 illustrates the 
image of Figure 6 after automatic correction of linear artifacts due to camera motion. 
This intermediate frame comprises a raster with a synthetic first raster field generated 
by duplicating the second raster field and then placing the synthetic first raster field into 
its corrected position. In an embodiment of the methods and system of the present 
invention, using the "Dyonics® Vision 625" system, such a synthetic field is generated 
and placed into its corrected position by automatically running the 
"ShiftEvenAndJoinOddField" function 504, as depicted in FIG. 5. The function 
"ShiftEvenAndJoinOdd Field" 504 replaces the even field in the NTSC frame at the 
offset determined by the function "MeasureMotionVector" 502. The following algorithm 
applies to a process of creating a raster with a synthetic field. 



15 void ShiftEvenAndJoinOddField(unsigned char •Fields, long x, long y, unsigned char *Rawlmage) 

unsigned char*Odd, *Even, *Workinglmage; 
long i, n, Lines, Skip, Shift, Width; 

20 // blindly fill both rasters with odd 
Workinglmage = Rawlmage; 
Even = Odd = Fields + 640*3 * 240; 
for(i=0; i<240; i++) 
{ 

25 n = 640*3; 

while(n-) 

*Workinglmage++ = *Even++; 

n = 640*3; 
while(n-) 

30 *Workinglmage++ = *Odd++; 

} 

// compute refill values... 
if(y>0) 
35 { 

Lines = 240-y/2; 
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Even = Fields; 

Workinglmage = Rawlmage + 2*640*3*(y/2); 
} 

else 

5 { 

Lines = 240+(y-1)/2; 

Even = Fields + 640*3*(-(y-1)/2); 

Workinglmage = Rawlmage; 

} 

10 if(y%2) 

{ 

Workinglmage += 640*3; // exchange 
} 

15 if(x>0) 

{ 

Skip = (640+x)*3; 
Shift = x*3; 
Width = (640-x)*3; 
20 Workinglmage += Shift; 

Q } 
uf% else 

m { 

~* Skip = (640-x)*3; 

S25 Shift = -x*3; 

~ Width = (640+x)*3; 

f" Even += Shift; 

y > 

L.30 // now join the rasters. . . 
4 for(i=0; i<Lines; i++) 

f { 

O n = Width; 

4= while(n-) 

C35 *Workinglmage++ = *Even++; 

O Workinglmage += Skip; 

Even += Shift; 

} 



4. Compute a subject motion map (Frame 3). 

Figure 8 illustrates image distortion due to camera motion and regional subject 
motion. After camera motion is automatically corrected according to the methods and 
45 system of the present invention, regional subject motion can then be automatically 
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detected and corrected, as shown at reference 406 in FIG. 4. Referring to FIG. 5, 
regional subject motion can be automatically detected by computing a subject motion 
map, which comprises another transparent, or non-viewable, intermediate frame, Frame 
3. Subject motion can be found anywhere in an image after camera motion effects are 
5 removed. The function "LocateArtifacts" 505, using the "Dyonics® Vision 625" system, 
for example, looks for the results of subject motion in the difference between even and 
odd rasters. To compute the difference between the "expected" even pixel value and 
the actual even pixel value, the "LocateArtifacts" 505 function uses reference images 
of the convolution kernel: 

So 

o 1 o 

T. 0 -20 : 

0 1 o. 

fj5 However, any equivalent convolution kernal could be used, such as: 

U 101 

« 0 -40 

S 101 

1S20 

IL or 

1 1 1 .v-.. 
0-60 

25 111. 

The kernal is chosen for speed of execution. The three active terms are 
adequate because the even raster of the image already has the greatest possible 
correlation to the odd raster because of the processes described above to create a 
30 raster with a synthetic field 504 and placing the synthetic field into its corrected position 
(frame 2). 
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"LocateArtifacts" 505 convolves the NTSC frame "ColorRaster" to produce a half 
height grayscale image in "Diff. The following algorithm applies to the process for 
computing a subject motion map. 



5 int absdif(int x, int y) 
{ 

int z; 

z = x-y; 
10 return z < 0 ? -z : z; 

} 

// color raster is 640x480x3 Diff is 640x240x1 
void LocateArtifacts(unsigned char *ColorRaster, unsigned char *Diff) 
15 { 

unsigned char Top, "Center, *Bottom; 
D int y, x, c; 
CI unsigned int Avg, DTB; 
Hi int Delta; 

So 

rU for(y=0; y<480; y += 2) 

yi { 

M= Top = ColorRaster + y * 640*3; 

M Center = Top + 640*3; 

£ 25 Bottom = Center + 640*3; 

£ x=64 °; 

while(x-) 

5 { 

^30 Delta = 0; 

9 c=3; 

while(c-) 
{ 

35 DTB = absdif(*Top, *Bottom); 

Avg = *Top++; 
Avg += *Bottom++; 
Avg /= 2; 

Delta += absdif(Avg,*Center++); 
40 Delta -= DTB/ 2; 

} 

Delta -= 15; 

if(Delta < 0) Delta = 0; 

Delta *= Delta; 

45 *Diff++ = Delta < 256 ? Delta : 255; 

} 

} 

} 



24 



Express Mail No. EL 146 846 537 US 

5. Establish a threshold difference and eliminate unwanted pixels in Frame 3 to 
create a binary subject motion location map (Frame 4). 

After a subject motion map is computed, a binary subject motion location map is 
generated to correct for regional motion artifacts by eliminating undesired pixels and yet 
preserve actual detail in the affected region. In the methods and system of the present 
invention, using the "Dyonics® Vision 625" system embodiment, for example, a binary 
subject motion location map is generated by the function "Decimate" 506 (FIG. 5). The 
function "Decimate" 506 compares each pixel in the grayscale image "map" to a 
threshold difference determined by experimenting with a large number of images. In 
practical application using the convolution kernal in "LocateArtifacts" 505, a threshold 
difference anywhere from about 20 to about 150 IRE brightness units works well. A 
preferred final average threshold difference value is 80 IRE units. 

"Decimate" 506 performs binary decimation on the subject motion map to 
eliminate hairline signals resulting from high image detail. Decimation eliminates 
unwanted pixels one at a time, and is performed by first counting the number of 
neighboring pixels exceeding the threshold. If more than three neighboring pixels in the 
subject motion map are above the threshold, the pixel is not decimated. If three or less 
neighboring pixels in the subject motion map are above the threshold, the pixel is 
eliminated. As such, local detail is preserved, while motion artifacts caused by surgical 
tool or patient movement is corrected. The following algorithm applies to the process of 
decimation to create a binary subject motion location map. 
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void Decimate(unsigned char *Map) 
{ 

int Total, Count = 640 * (240 - 2) - 2; 
unsigned char *Previous, *Current, *Next; 
5 unsigned char Threshold = 80; 

Previous = Map; 
Current = Previous + 640; 
Next = Current + 640; 
10 while(Count-) 
{ 

Total = 0; 

Total += Previous[-1] < Threshold ? 0 : 1 ; 

Total += Previous[0] < Threshold ? 0 : 1; 
15 Total += Previous[1 ] < Threshold ? 0 : 1 ; 

Total += Current[-1] < Threshold ? 0 : 1; 

Total += Current[1] < Threshold ? 0 : 1; 

Total += Next[-1] < Threshold ? 0 : 1 ; 

Total += Next[0] < Threshold ? 0 : 1; 
20 Total += Next[1] < Threshold ? 0 : 1 ; 

"Current = Total < 3 ? 0 : "Current; // leave alone 
Previous++; 
Current++; 
25 Next++; 
} 

} 



6. Adjust the binary subject motion location map (Frame 5). 

30 After establishing a threshold and decimating unwanted pixels to create a binary 

subject motion location map in intermediate frame 4 in the previous step, the binary 
map image is adjusted in a subsequent intermediate frame to replace decimated areas 
but without the blurring caused by motion artifacts. In the methods and system of the 
present invention, using the embodiment of the "Dyonics® Vision 625" system, for 

35 example, the map is "dialated" to identify pixels near the boundaries of regions 
eliminated due to subject motion artifacts. Pixels near the boundaries are most likely to 
be affected by similar subject motion. The "Dialate" routine 507 (FIG. 5) replaces 
decimated areas but without the blurring caused by regional motion artifacts. The 
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"Dialate" routine 507 works in similar fashion to the "Decimate" routine 506, except the 
effect is to "spread" the map out one pixel each time the routine is called. A two- 
dimensional vector is computed by detecting pixels in two directions, one pixel at a 
time, adjacent to pixels identified at the boundaries of eliminated regions. Pixels 
5 eliminated within regions of subject motion are then replaced with the two-dimensional 
vector to create a corrected image having smooth edges. The "Dialate" routine 507 is 
automatically run twice to enhance image sharpness. The following algorithm applies 
to the process for adjusting the map at pixels near the threshold. 



10 void Dialate(unsigned char *Map) 
{ 

int Total, Count = 640 * (240 - 2) - 2; 
unsigned char 'Previous, 'Current, *Next; 
unsigned char Threshold = 80; 

15 

Previous = Map; 
Current = Previous + 640; 
Next = Current + 640; 
while(Count-) 
20 { 



Total = 0; 



25 



Total += Previous[-1] < Threshold ? 0 : 1 ; 
Total += Previous[0] < Threshold ? 0 : 1; 
Total += Previous[1] < Threshold ? 0 : 1; 
Total += Current[-1] < Threshold ? 0 : 1; 
Total += Current[1] < Threshold ? 0 : 1; 
Total += Nextl-1] < Threshold ? 0 : 1; 



Total += Next[0] < Threshold ? 0 : 1; 
Total += Next[1] < Threshold ? 0 : 1; 



30 



35 



*Current = Total > 4 ? 255 : "Current; // leave alone 

Previous++; 

Current++; 

Next++; 

} 



7. 



Compute a finished, corrected image (Frame 6). 
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After the binary subject motion location map image is adjusted by replacing 
decimated areas but without blurring caused by regional motion artifacts, a finished, 
corrected image is computed. A finished, corrected image is computed by using the 
adjusted map (frame 5) to indicate where to modify the original NTSC frame 

5 ColorRaster. In the methods and system of the present invention, using the 
embodiment of the "Dyonics® Vision 625" system, for example, the original frame 
ColorRaster is modified by running the "InterpolateMap" routine 508, as shown in FIG. 
5. The "InterpolateMap" routine 508 selectively corrects the ColorRaster by computing 
an even raster from the odd raster where the map indicates subject motion is greatest. 

10 That is, local regions of the image field having large amounts of subject motion are 
selected out for correction. As a result, a sharp finished image is interpolated. FIG. 9 
illustrates the image of Figure 8 after automatic motion correction. The following 
algorithm applies to a process of selectively correcting the original frame ColorRaster to 
produce a finished image, corrected for both camera motion and regional subject 

15 motion. 



// color raster is 640x480x3 Map is 640x240x1 

void lnterpolateMap(unsigned char 'ColorRaster, unsigned char *Map) 

{ 

20 unsigned char Top, "Center, *Bottom; 
int y, x, c; 

unsigned int Avg, DTB; 
int Delta; 

25 Top = ColorRaster; 
Center = Top + 640*3; 
Bottom = Center + 640*3; 

y = 240; 
30 while(y-) 
{ 

x = 640; 
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while(x-) 
{ 

if(*Map++ > 128) 
{ 

Avg = *Top++; 
Avg += *Bottom++; 
*Center++ = Avg » 1; 
Avg = *Top++; 
Avg += *Bottom++; 
*Center++ = Avg » 1; 
Avg = *Top++; 
Avg += *Bottom++; 
*Center++ = Avg » 1; 

} 

else 

{ 

Top += 3; 
Center += 3; 
Bottom += 3; 
} 

} 

Top += 640 * 3; 
Center += 640 * 3; 
Bottom += 640 * 3; 

} 

} 



As a result of operation of steps 505-508 in FIG. 5 (frames 3-6), the original 
captured frame (frame 1) is automatically corrected for regional subject motion, as 
indicated by step 406 in FIG. 4. 



8. Display the finished, corrected image (Frame 7). 

After a finished image 509 (FIG. 5), automatically corrected for both camera 
motion and regional subject motion, is computed, the image is displayed for real-time 
viewing, by the surgeon for example. Images automatically corrected according to the 
methods and system of the present invention, using the embodiment of the "Dyonics® 
Vision 625" system, for example, may also be stored in standard image file format on 
transportable media for long term archival, retrieval, or printing. Referring to FIGS. 1 
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and 3, as a finished, corrected image (frame 7) is displayed, the image is saved (309) in 
the internal temporary storage 112, such as an internal cache, of the digital capture unit 
103. Corrected images saved in the internal temporary storage 112 may then be 
written (310) by a media writer 106 for permanent storage onto portable storage media 
107. Corrected images may also be printed using a standard video printer 108. 

The above embodiment of the present invention has been described in fulfillment 
of the various objects of the invention. It should be recognized that this embodiment is 
merely illustrative of the principles of the present invention. Numerous modifications 
and adaptations thereof will be readily apparent to those skilled in the art without 
departing from the spirit and scope of the present invention. For example, methods and 
systems of automatic motion correction of the present invention may be utilized in a 
variety of video signal applications and with alternative computer hardware modalities. 
As another example, although algorithms are provided herein that apply to various 
steps according to the present invention, other algorithms may be employed. 
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What is claimed is: 

1 . A method for automatic correction of motion artifacts in an interlaced video 
image captured by an image recording camera, comprising: 

capturing a complete frame of an interlaced video image, the complete frame 
5 having a first raster field and an interlaced second raster field; 
automatically correcting for camera motion; 
automatically correcting for subject motion; and 
displaying an image corrected for camera motion and subject motion. 



10 2. The method of claim 1 , wherein automatically correcting for camera motion 
comprises determining whether the captured frame contains camera motion artifacts. 

3. The method of claim 2, wherein automatically correcting for camera motion 
comprises performing auto-correlation on the first raster field with respect to the second 
15 raster field. 



4. The method of claim 3, the first and second raster fields each having a plurality 
of pixels and pixels in the first raster field are offset from pixels in the second raster 
field, wherein performing auto-correlation comprises creating a two-dimensional motion 
20 vector between pixels in the first raster field and pixels in the second raster field. 
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5. The method of claim 4, wherein creating a two-dimensional motion vector 
comprises locating optimal correlation values for X/Y coordinates for each pixel in the 
first raster field relative to a reference pixel in the second raster field. 

5 6. The method of claim 5, wherein locating optimal correlation values comprises 
using a repeating 3x3 convolution search. 

7. The method of claim 6, wherein using a repeating 3x3 convolution search 
comprises 

10 (a) determining a first correlation value for corresponding pixels in the first and 

second raster fields when a first specified offset is zero for both X and Y coordinates, 
(b) determining a second correlation value for a pixel in the first raster field to the 

right of the corresponding pixel in the second raster field when a second specified offset 

is one for X and zero for Y, 
is (c) calculating a difference between the first correlation value and the second 

correlation value, 

(d) squaring the difference between the values, 

(e) repeating steps (a), (b), (c), and (d) for all pixels in the first raster field relative 
to pixels in the second raster field, 

20 (f) adding the squares of the differences between correlation values at the first 

specified offset and at the second specified offset, and 
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(g) determining the correlation values which produce a minimum difference 
between pixels in the first raster field and the second raster field to provide optimal 
correlation values for shifting the first raster field relative to the second raster field. 

5 8. The method of claim 4, wherein creating a two-dimensional motion vector 
comprises locating values for X/Y coordinates for each pixel in the first raster field 
determined to be offset more than 15 pixels from a reference pixel in the second raster 
field. 

10 9. The method of claim 3, wherein automatically correcting for camera motion 
further comprises creating a synthetic first raster field by duplicating the second raster 
field. 

1 0. The method of claim 4, wherein creating a synthetic first raster field by 

15 duplicating the second raster field comprises replacing the first raster field with the 
duplicated second raster field in the captured complete frame in a corrected position 
according to the auto-correlation determined by the two-dimensional motion vector. 

1 1 . The method of claim 1 , wherein automatically correcting for subject motion 

20 comprises computing a subject motion map to automatically identify regions of subject 
motion in the captured frame. 
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12. The method of claim 1 1 , wherein computing a subject motion map comprises 

(a) determining actual pixel values for the first raster field, 

(b) computing predicted pixel values for the first raster field from the second 
raster field, 

(c) comparing the predicted pixel values and the actual pixel values for the first 
raster field to determine differences between the first and second raster fields in 
discrete regions of the captured frame, 

(d) identifying regions of the captured frame where differences between the first 
and second raster fields are relatively large, and 

(e) squaring the relatively large differences between the first and second raster 
fields to generate the subject motion map. 

1 3. The method of claim 12, wherein computing a subject motion map further 
comprises convolving the first and second raster fields of the captured frame to produce 
a half-height grayscale image map in regions of large differences in the subject motion 
map, and leaving uncorrected regions of the captured frame where differences between 
the first and second raster fields are relatively small. 

1 4. The method of claim 1 1 , wherein automatically correcting for subject motion 
further comprises creating a binary subject location map to delineate regions of the 
captured frame for applying correction for subject motion. 
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15. The method of claim 14, wherein creating a binary subject location map 
comprises 

(a) establishing a threshold difference between the predicted pixel values and 
the actual pixel values for the first raster field, 
5 (b) comparing each pixel in the grayscale image map to the threshold difference, 

counting the number of pixels exceeding the threshold difference, 

(c) eliminating from the grayscale image map pixels where three or less 
neighboring pixels in the grayscale image map are above the threshold difference, and 

(d) leaving in the grayscale image map pixels where more than three 
10 neighboring pixels are above the threshold difference. 

16. The method of claim 1 5, wherein the threshold difference is in the range from 
about 20 to about 150 IRE brightness units. 

15 17. The method of claim 15, wherein the threshold difference is 80 IRE brightness 
units. 

18. The method of claim 15, wherein automatically correcting for subject motion 
further comprises adjusting the binary subject location map by replacing pixels 
20 eliminated from the grayscale image map in regions of subject motion. 
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19. The method of claim 18, the regions of subject motion having pixels eliminated 
forming boundaries comprising pixels, wherein adjusting the binary subject location 
map comprises 

(a) computing a two-dimensional vector from pixels at the boundaries of 
5 eliminated regions of subject motion, 

(b) replacing pixels eliminated from regions of subject motion with the two- 
dimensional vector, and 

(c) repeating steps (a) and (b) by computing the two-dimensional vector at 
locations one pixel further away from the boundaries of the eliminated regions of 

10 subject motion to create a corrected image having smooth edges. 

20. The method of claim 19, wherein computing a two-dimensional vector from pixels 
at the boundaries of eliminated regions of subject motion comprises identifying 
boundaries of eliminated regions of subject motion, and detecting pixels in two 

15 directions, one pixel at a time, adjacent to the pixels at the boundaries. 

21 . The method of claim 19, wherein automatically correcting for subject motion 
further comprises computing a finished, corrected image. 

20 22. The method of claim 21 , wherein computing a finished, corrected image 
comprises using the adjusted map to indicate regions on the captured frame where 
subject motion is greatest, and computing a corrected second raster field from a 
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corrected first raster field in regions on the captured frame where subject motion is 
greatest. 

23. The method of claim 22, further comprising displaying the finished image 
5 corrected for camera motion and subject motion. 

24. The method of claim 1 , further comprising automatically correcting for subject 
motion after automatically correcting for camera motion. 

fi io 25. The method of claim 1 , wherein capturing the complete frame of the interlaced 
J, video image comprises capturing video images taken during surgical procedures. 

^ 26. A method for automatic correction of motion artifacts in an interlaced video 
% image captured by an image recording camera, comprising: 
A 15 capturing a complete frame of an interlaced video image, the complete frame 

having a first raster field and an interlaced second raster field, the first and second 
raster fields each having a plurality of pixels; 

locating optimal correlation values between pixels in the first raster field and 
pixels in the second raster field; 
20 creating a two-dimensional motion vector from optimal correlation values; 
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creating a synthetic first raster field by duplicating the second raster field in the 
captured complete frame in a corrected position according to the two-dimensional 
motion vector; 

computing a subject motion map to identify regions of the captured frame where 
5 differences in pixel values between the first and second raster fields are relatively large; 

creating a binary subject location map to delineate regions of the captured frame 
for applying correction for subject motion; 

eliminating from the binary subject location map pixels where the number of 
neighboring pixels exceeds a pre-determihed threshold; 
10 adjusting the binary subject location map by replacing eliminated pixels; 

computing a finished, corrected image; and 

displaying the image corrected for camera motion and subject motion. 

27. A system for automatic correction of motion artifacts in a live, interlaced video 
15 image, the system comprising: 

an image recording camera for capturing complete frames of video images; 

a digital capture unit for processing live video images and captured frames of 
video images; 

a first filter for automatically correcting for camera motion; 
20 a second filter for automatically correcting for subject motion; and 

a video monitor for displaying images. 
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28. The system of claim 27, the complete frames each having a first raster field and 
an interlaced second raster field, each field comprising a plurality of pixels, wherein the 
first filter for automatically correcting for camera motion comprises 

a two-dimensional motion vector between the first and second raster fields 
5 created by auto-correlation, and 

a synthetic first raster field created by duplicating the second raster field in a 
corrected position in the captured complete frame according to the two-dimensional 
motion vector. 

29. The system of claim 28, wherein the second filter for automatically correcting for 
subject motion comprises 

a subject motion map computed to identify regions of subject motion, 
a binary subject motion map for eliminating pixels in the regions of subject 
motion, 

an adjusted binary subject motion' map, the binary subject motion map adjusted 
by replacing eliminated pixels, and 

a corrected captured frame, the frame corrected by computing a corrected 
second raster field from the first raster field in regions where subject motion is greatest. 

20 30. The system of claim 27, wherein the video monitor for displaying images 
comprises images displayed before and after correction for camera motion and for 
subject motion. 
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31 . The system of claim 27, wherein the system further comprises a freeze mode for 
freezing live video images and displaying frozen images on the video monitor. 

32. The system of claim 31 , wherein the system further comprises a capture mode 
for digitally capturing frozen images by the digital capture unit. 

33. The system of claim 32, the digital capture unit having an internal temporary 
storage capacity, wherein the system further comprises a save mode for saving images 
corrected for camera motion and subject motion in the internal temporary storage of the 
digital capture unit 

34. The system of claim 33, the system having a media writer for permanently saving 
images onto portable storage media, wherein the system further comprises a write 
mode for permanently saving images corrected for camera motion and subject motion 
onto portable storage media. 
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ABSTRACT 

Methods and systems of the present invention automatically and differentially 
detect and correct linear motion artifacts caused by camera movement and regional 
subject motion artifacts. Regional subject motion artifacts may be caused, for example 
in endoscopic or other surgery, by movement of surgical tools or the patient within the 
image field. Methods and systems of the present invention automatically correct for 
both types of motion occurring simultaneously. After an image is automatically 
corrected for both camera motion and regional subject motion, the image is displayed 
for viewing. 



41 



Endoscope 
Optic 

Video Out 



-161 



Video In 




Camera 




Control 


Video 


Unit 


Out 



-{02. 



Freeze 



Signal 



Capture 



Signal 



Video In 



AMC 
Filter 



|0<? 



Media Writer 



107 



Storage 
Media 




113 



BYPASS 



Digital Capture 
Unit 



Temp. Storage 



Display Live 



Display Still 



103 



[0% 



Video 
Monitor 



100 



Video Printer T__- !6 g 



FIGURE 1 



zoo 

MAIN MENU 




New Patient 
Menu 



Video Signal Mode 

Time 

Date 

^Language 
^MC Filter 
fi>SD Status 




2J>2 



Patient Name 
Patient ID 
Surgeon 
Operative Site 



Freeze 
Capture 
OSD Status 
Video Signal Mode 
Write Saved Images 
to Media 



Write Saved 
I mages to 
Media 



View Live Images 
Without Capture 



Save Images 
to Media 



FIGURE 2 



Activate Digital Capture Unit (DCU) 



Select video signal mode 



Activate AMC Filter fz_ 



Enter new patient data 




View live video images 



FREEZE OFF 



DISPLAY 
frozen images on monitor 



307 



CAPTURE 
frozen images by DCU 



303 



SAVE 
images temporarily to 
DCU internal cache 



WRITE 
saved images to 
portable media 



•320 



261 



102. 



303 



BYPASS 
digital capture 



BYPASS OFF 



Return to view live video images 



*1 



loSl 



FIGURE 3 



FREEZE 



fa/ 



DISPLAY 



4oz 



CAPTURE 



4o3> 



Capture complete frame 



Automatic correction for camera motion 



Automatic correction for regional subject motion 



Display finished, corrected image 



SAVE 



■408 



WRITE 



FIGURE 4 



FRAME 1 



FRAME 2 



FRAME 3 



FRAME 4 



FRAME 5 



FRAME 6 



FRAME 7 



Capture complete frame 



MeasureMotionVector 
Even/odd field motion vector 



J 



Convolution 

Optimum raster auto-correlation 



ShiftEvenAndJoinOddField 
Synthetic raster field 



LocateArtifacts 
Subject motion map 



Decimate 

Eliminate unwanted pixels 



sol 



563 



SOS 



S<36 




SOI 



InterpolateMap 

Compute even raster from odd raster 



508 



Display finished corrected image 



FIGURE 5 




FIGURE 6 




FIGURE 7 



1 t , ► 




FIGURE 8 




FIGURE 9 



